| 1 |
Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost
|
|
|
|
In: ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics ; https://hal.archives-ouvertes.fr/hal-03613101 ; ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, May 2022, Dublin, Ireland (2022)
|
|
BASE
|
|
Show details
|
|
|
| 2 |
Imputing out-of-vocabulary embeddings with LOVE makes language models robust with little cost
|
|
|
|
In: ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics ; https://hal.archives-ouvertes.fr/hal-03613101 ; ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, May 2022, Dublin, Ireland (2022)
|
|
Abstract:
International audience ; State-of-the-art NLP systems represent inputs with word embeddings, but these are brittle when faced with Out-of-Vocabulary (OOV) words. To address this issue, we follow the principle of mimick-like models to generate vectors for unseen words, by learning the behavior of pre-trained embeddings using only the surface form of words. We present a simple contrastive learning framework, LOVE, which extends the word representation of an existing pre-trained language model (such as BERT), and makes it robust to OOV with few additional parameters. Extensive evaluations demonstrate that our lightweight model achieves similar or even better performances than prior competitors, both on original datasets and on corrupted variants. Moreover, it can be used in a plug-and-play fashion with FastText and BERT, where it significantly improves their robustness.
|
|
Keyword:
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; Language models; Out-of-vocabulary OOV words; Word embeddings
|
|
URL: https://hal.archives-ouvertes.fr/hal-03613101
https://hal.archives-ouvertes.fr/hal-03613101/document
https://hal.archives-ouvertes.fr/hal-03613101/file/Imputing%20OOV%20Embeddings%20with%20LOVE.pdf
|
|
BASE
|
|
Hide details
|
|
|
| 3 |
Population modeling with machine learning can enhance measures of mental health
|
|
|
|
In: ISSN: 2047-217X ; GigaScience ; https://hal.inria.fr/hal-03470466 ; GigaScience, BioMed Central, 2021, ⟨10.1101/2020.08.25.266536⟩ (2021)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Population modeling with machine learning can enhance measures of mental health
|
|
|
|
In: Gigascience (2021)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Exploring the anatomical encoding of voice with a mathematical model of the vocal system.
|
|
|
|
In: ISSN: 1053-8119 ; EISSN: 1095-9572 ; NeuroImage ; https://hal.inria.fr/hal-01498364 ; NeuroImage, Elsevier, 2016, 141, pp.31-9. ⟨10.1016/j.neuroimage.2016.07.033⟩ (2016)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Identification of Mood-Relevant Brain Connections Using a Continuous, Subject-Driven Rumination Paradigm
|
|
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Identification of Mood-Relevant Brain Connections Using a Continuous, Subject-Driven Rumination Paradigm.
|
|
|
|
In: ISSN: 1047-3211 ; EISSN: 1460-2199 ; Cerebral Cortex ; https://hal.inria.fr/hal-01094759 ; Cerebral Cortex, Oxford University Press (OUP), 2014, pp.12 (2014)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
API design for machine learning software: experiences from the scikit-learn project
|
|
|
|
In: European Conference on Machine Learning and Principles and Practices of Knowledge Discovery in Databases ; https://hal.inria.fr/hal-00856511 ; European Conference on Machine Learning and Principles and Practices of Knowledge Discovery in Databases, Sep 2013, Prague, Czech Republic (2013)
|
|
BASE
|
|
Show details
|
|
|
| 9 |
Decoding Visual Percepts Induced by Word Reading with fMRI
|
|
|
|
In: Pattern Recognition in NeuroImaging (PRNI), 2012 International Workshop on ; https://hal.inria.fr/hal-00730768 ; Pattern Recognition in NeuroImaging (PRNI), 2012 International Workshop on, Jul 2012, Londres, United Kingdom. pp.13-16, ⟨10.1109/PRNI.2012.20⟩ ; http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6295916&tag=1 (2012)
|
|
BASE
|
|
Show details
|
|
|
|
|
